Quality Anomaly Detection Using Predictive Techniques: An Extensive Big Data Quality Framework for Reliable Data Analysis

The increasing reliance on Big Data analytics has highlighted the critical role of data quality in ensuring accurate and reliable results. Consequently, organizations aiming to leverage power recognize crucial as an integral component. One notable type anomaly observed big datasets is presence outlier values. Detecting addressing these outliers have become a subject interest across diverse domains, leading development numerous detection approaches. Although witnessed proliferation practices recent years, significant gap remains anomalies related other aspects quality. Indeed, while most approaches focus identifying that deviate from expected patterns, they do not consider irregularities quality, such missing, incorrect, or inconsistent data. Moreover, are domain-correlated lack capability detect generic manner. Thus, we aim through this paper address field provide holistic effective solution for detection. To achieve this, suggest novel approach allows comprehensive six dimensions: Accuracy, Consistency, Completeness, Conformity, Uniqueness, Readability. framework sophisticated implementation intelligent model without any correlation specific field. Furthermore, introduce measure new metric called “Quality Anomaly Score,” which refers degree anomalousness each dimension entire dataset. Through evaluation our framework, suggested achieved accuracy score up 99.91% F-score 98.07%.